Leaving Free WP All Over the Field

If you were a coach, would you voluntarily give up a down at some point in the game, just to be sporting? Ehh, let's just make it 3rd and 5 instead of 2nd and 5. Of course not. For a random play in the 2nd quarter, that would cost you about 0.02 WP (2% chance of winning) for no reason.

So why do NFL coaches voluntarily leave WP out on the field?

Take yesterday's DEN-SEA game as an example. SEA was ahead 17-12 in the 4th quarter, and had the ball deep in their own territory with about 9 minutes to play. With the game clock running, they snapped the ball with: 8, 5, 5, 8, and 10 seconds left on the play clock. That's a total of 36 seconds. Plus, there was a play in which the receiver could have just as easily remained in bounds. Because there was more than 5 minutes left in the game and the clock restarts after the ball is set, that may have only cost 10-15 seconds of play clock rather than up to 40 seconds. To be fair let's say there was a total of 46 seconds SEA could have burned off the clock during their second to last drive with almost no effort or risk.

Texans Try Once, Fail Twice

Down 14-0 at the start of the second half to the New York Giants,
the Houston Texans faced a 4th-and-1 on their own 46-yard line. At this point, with just a 9.0% chance to win, Bill O'Brien made the correct call to go for it. A successful conversion means a 12.9% win probability, while a punt means about an 8.6% chance to win. The break-even point going for it is far below an estimated 65% conversion rate on 4th-and-1. Alfred Blue ran off right tackle and was stuffed, turning the ball over on downs. The Giants would kick a field goal to go up 17-0.

On the very next drive, Ryan Fitzpatrick led the Texans downfield to the Giants 9-yard line where they faced another 4th-and-1. With 6:13 left in the third, down 17, Bill O'Brien elected to take the chip-shot field goal. Even the commentators suggested he should be going for it. Obviously, the prior failure on fourth down should not have an affect on the Texans' decision this time. If that were the case, O'Brien would be judging his previous decision on the outcome, rather than the process. The only other logic could be that he figured they would need a field goal at some point, down 17 - common faulty logic in the NFL as coaches should be doing whatever they can to maximize their chances of winning.

Two-Point Conversion in the KC-DEN game

With 7:15 left in the 4th quarter against DEN, KC's Knile Davis ran for a 4-yard TD, narrowing the Broncos' lead 21-16 pending the extra point or two-point conversion. Andy Reid elected for the extra point, and following the kick the Chiefs trailed by 4 points rather than 3 or 5 points resulting from a two-point try.

NFL coaches typically adhere to what's known as the Vermeil Chart for making two-point decisions. The chart was created by Dick Vermeil when he was offensive coordinator for UCLA over 40 years ago. It's a very simple chart that simply looks at score difference prior to any conversion attempt and does not consider time remaining, with one caveat. It applies only when the coach expects there to be three or fewer (meaningful) possessions left in the game.

With just over 7 minutes to play, there could be three possessions at most left, especially considering that at least one of those possessions would need to be a KC scoring drive for any of this to matter. (In actuality, there were only two possessions left, one for each team.) Even the tried-and-true Vermeil chart says go for two when trailing by 5. But it's not the 1970s any more and this isn't college ball, so let's apply the numbers and create a better way of analyzing go-for-two decisions.

Except for rare exceptions I've resisted analyzing two-point conversion decisions with the Win Probability model because, as will become apparent, the analysis is particularly susceptible to noise. Now that we've got the new model, noise is extremely low, and I'm confident the model is more than up to the task.

First, let's walk through the possibilities for KC intuitively. If KC fails to score again or DEN gets a TD, none of this matters. Otherwise:

Chiefs Crawling Drive, Come Away With Nothing

The Chiefs lost to the Broncos 24-17 on Sunday and had a chance to at least tie the game at the very end. Kansas City kept Peyton Manning off the field for an enormous chunk of the second half. The Broncos offense had only two drives after halftime (not including the final kneel down), one for a punt, one for a field goal, totaling just 8:51 in possession. The longest drive came from the Chiefs at the very start of the second half, where they ran 23 plays, taking 10 minutes off the clock... and ultimately missed a field goal. This got me thinking, how does drive length (in minutes) affect the probability of a team scoring?

First, here's a look at the ridiculous drive using our Markov model:

Analyzing Replay Challenges

The new WP model allows some nifty new applications. One of the more notable improvements is the consideration of timeouts. That, together with enhanced accuracy and precision allow us to analyze replay challenge decisions. Here at AFA, we've tinkered with replay analysis before, and we've estimated the implicit value of a timeout based on how and when coaches challenge plays. But without a way to directly measure the value of a timeout the analysis was only an exercise.

Most challenges are now replay assistant challenges--the automatic reviews for all scores and turnovers, plus particular plays inside two minutes of each half. Still, there are plenty of opportunities for coaches to challenge a call each week.

The cost of a challenge is two-fold. First, the coach (probably) loses one of his two challenges for the game. (He can recover one if he wins both challenges in a game.) Second, an unsuccessful challenge results in a charged timeout. The value of the first cost would be very hard to estimate, but thankfully the event that a coach runs out of challenges AND needs to use a third is exceptionally rare. I can't find even a single example since the automatic replay rules went into effect.

So I'm going to set that consideration aside for now. In the future, I may try to put a value on it, particularly if a coach had already used one challenge. But even then it would be very small and would diminish to zero as the game progresses toward its final 2 minutes. In any case, all the coaches challenges from this week were first challenges, and none represented the final team timeout, so we're in safe waters for now.

Every replay situation is unique. We can't quantify the probability that a particular play will be overturned statistically, but we can determine the breakeven probability of success for a challenge to be worthwhile for any situation. If a coach believes the chance of overturning the call is above the breakeven level, he should challenge. Below the breakeven level, he should hold onto his red flag.

Weekly Game Probabilities: A New Home

This season the weekly game probabilities will be featured at Sports on Earth. Each game will have the probability, a score prediction, and a couple notes on why the numbers are what they are. In the early weeks of the season, the numbers are at least partially based on the same preseason estimates of team strength I used for the season projections. But as we get a few weeks of data, those preseason ratings will fade out.

For now the score predictions are simply maximum-plausibility estimates. (Yes, I just made that term up.) Predicting an actual score for each game is statistically boring. With few exceptions, a statistically sound estimate would be 24-20 or 27-21 for every game, so I've added some of the human element to the score predictions. The bottom line is that readers should focus on the probabilities and don't bet the mortgage on the scores.

The game probabilities will be matched up against the picks of Will Lietch, one of the cornerstone writers at SOE. The idea is to create a friendly competition between man and machine.

The game probabilities had a great run at the New York Times--5 years. But there are only so many thought-provoking or counter-intuitive lessons on probabilities and predictions that can be squeezed out of a week of NFL games. But AFA will continue working with the Times on various projects as the season unfolds.

Here's the link to the probabilities for week one. For those keeping score at home, I had the Seahawks at 66% to win last night.

The 4th Down Bot Returns

The 4th Down Bot is returning to the New York Times this season. You might recall we booted him up late last season, but this year he'll be around starting week one. At its heart, the bot is a fun application of the 4th Down Calculator feature here at AFA. It uses both the Expected Points model and the Win Probability model to estimate the best option for every 4th down as a game is in progress.

As I mentioned last year, although the 4th down issue is growing mold with smarter fans, it remains the lowest hanging fruit on the football analytics tree. So it's nice to be able to automate things and not have to do the analysis myself. But on the other hand, we can add 'football analyst' to the list of jobs being taken over by robots.

The Bot will be faster, more accurate, and come with some new features this season. Here is a brief introduction. Here's are a few notes on how it works. And here is his Twitter feed.

Super Bowl XVLXVLVLIIICDMVXXXIII Analysis

First of all, I'm getting tired of the Roman numerals. It was cool up until maybe Super Bowl XXV, but now it just hurts my brain.

Secondly, although my numbers pointed to a SEA edge I did not see that coming. The game notched a 1.5 on the Excitement Index, the lowest of any SB in the data (since '99). The next lowest were the TB-OAK 2002 game and the BAL-NYG 2000 game, each at 2.7. There weren't many decisions to analyze because the game got out of hand so quickly, but I'll go over the little we can learn from last night.

Overall, the game hinged on the fundamentals. SEA's defense was faster, bigger, stronger. Even a layman like myself could tell SEA won because of lots and lots of individual matchup victories. They made tackles at first contact. Guys shook their blocks lightning fast. They swarmed to the screens, caved the pocket, and covered the receivers in stride. There weren't many blitzes or scheming contrivances. Instead it was plain old physical football. The only wrinkle I noticed was that SEA played more cover/man 2 than we expected, but that's not exactly something Manning shouldn't normally be able to handle.

The Challenges

Advanced Stat Breakdown for Super Bowl 48

Instead of reading a bunch of words about the Super Bowl matchup, where each phase is trying its hardest to express some sort of numerical evaluation, wouldn't you prefer to see the numbers themselves collected into one giant eye chart? Well, if that appeals to you, you'll enjoy the SEA-DEN Matchup page.

NFCCG SF-SEA Observations

As expected, this was a real defensive slugfest. The winning QB had -3.4 EPA. Kaepernick posted -0.28 WPA and 2.2 AYPA. Both offensive lines were beaten soundly. SF's notched -5.4 EPA and SEA's had -2.6 EPA.

Unlike the AFC game, this one was all about 4th downs. HUGE leverage throughout the game. I know I can be a broken record on this stuff, but this game really hinged on some very interesting strategic decisions.

-SF 4th and 2 on the SEA 7, 1st qtr. They punted. Probably should have gone for it.

-SF 4th and goal on the SEA 1. They went for it. Great call.

-SF 4th and 6 on the SEA 46, 1st qtr. They punted. Probably should have gone for it.

-SEA 4th and 6 on SF 38, 26 sec in 2nd qtr. They went for it, converted then kicked a FG to end the half.

-SEA 4th and 7 on the SF 35, 4th qtr 14 min to play, down by 4. They went for it. Great call except SEA burned a timeout that they were reasonably likely to need in order to think things over. Here's the thing: Timeouts are very valuable. If you can't decide between going for it or kicking or punting, you're probably very close to the point of indifference anyway. You may be better off making any quick decision and saving the timeout than you are making an optimum decision but wasting a timeout.

-SEA 4th and goal from the 1, 4th qtr 8:39 to play, up by 1. The went for it. Great call. Why? First, because they'll probably make it and virtually put the game away. And if they don't they're likely to leave the ball on the SF 1-yd line. That's not exactly a good place to be for an offense. I heard someone say that despite the math you can't take a chance like that against the SF defense. But as I noted last week, over the past 2 seasons SF has faced 15 (now 17) plays from the 1-yd line and allowed TDs on 10 of them. That's worse than league average. Don't get me wrong. I'm not saying that the SF defense is below average. Instead, the point is that good and bad teams aren't that different on any one given play. It's just that good or bad teams show up that way after accumulating very small advantages over several dozen individual plays in a game.

-Here's a weird one. SEA 4th and 11 on the SF 29, 4th qtr 3:43 to play, up by 3. My numbers say...punt? Yes, punt. Here's why:

AFCCG NE-DEN Observations

I thought the big story of the game wasn't how easily DEN moved the ball. We all expected that. The big story was DEN's defense, which held NE to just 3 points in the 1st half and 10 points through 55 minutes. Brady was held to -0.02 WPA. He did notch +8.3 EPA, but a lot of that was after the game was mostly decided.

NE was going to need some fluky things to go their way to win--turnovers, a special teams play, or some terrible call by the refs. It never came.

Manning and the DEN passing game did have a fantastic day. Manning: +.48 WPA, +17.9 EPA, 9.3 AYPA, no turnovers, no sacks.

I, and the NYT 4th Down Bot--(funny how you never see the two of us together at the same time), agreed with every 4th down call during the game. Belichick knows what he's doing. I was disappointed to see DEN burn a timeout just prior to NE's 4th down conversion attempt. Teams should be better prepared for a 4th down attempt, particularly in situations like this: a 4th and short in or near the red zone. In a high-leverage situation like that, it's ok for a team with a significant lead to use a timeout, but in a closer game, it would be much more costly. (I'm working on a project to value timeouts in terms of WP now, and without any spoilers--they are very precious in the 2nd half.)

Seahawks Should Not Have Scored

I'm not going to touch Sean Payton's late field goal attempt, down eight, into the wind, with only one timeout remaining. After the miss, on the ensuing Seahawks drive, Marshawn Lynch tore down the left sideline and it appeared he was let into the end zone by the Saints. The question is, did the Saints make the right decision letting him score once the first down was achieved - and should Lynch have gone down at the 1-yard line? The answer to both of these will always be the same, and in this case, it's yes.

Lynch scored with 2:48 remaining in the game. Up by 15, kicking off to the Saints, the Seahawks had about a 99% chance of winning. The Saints would need to score, recover an onside kick, and score again just to tie.

Update: Lynch actually scored with 2:40 remaining, not 2:48 as originally listed in the play-by-play. That means that the Seahawks could have milked the entire clock as listed at the very bottom of this article.

But, what if Lynch had gone down at the one? Using the time calculator, we can see how much time the Saints would have had following a chip shot (99.7%) field goal from the 1-yard line. The Saints had no timeouts left, so we are looking at the blue line.

Would Auburn Have Been Better Off Not Scoring a Touchdown?

Slate asked me to take a look at the possibility that Auburn would have a higher win probability had they not scored on Tre Mason's go-ahead touchdown run, and instead taken a knee at the 1-yard line with 1:19 to play. It was a difficult analysis, and required some unsatisfying assumptions, but in the end the results confidently pointed toward one conclusion.

...Nevertheless, Auburn had about an 81 percent chance of winning after Mason’s score, as teams in Florida State’s position are able to score a touchdown about 19 percent of the time. (These numbers are based on analogous situations in the NFL, though I’ve made slight adjustments for the differences in pace between college and the pros, and to include the chance of a kick return for a touchdown.)

So, the question we’re evaluating is whether having a first down at the 1-yard line would have given Auburn more than an 81 percent chance of winning. It’s a tricky question because it needs to be analyzed backwards...

(What I didn't explain in the article is that it's easiest to work backwards because the Auburn WP on 3rd down is depending on the results of a potential 4th down. And the WP on 2nd down are dependent on the potential results of 3rd down, which in turn depend on 4th down. And so on.)

For the Slate articles, I can't get away as much math and equations as I like, so here's a table of the relevant probabilities I used. It was complicated because the deeper into the goal-line series Auburn went, the lower Auburn's chances of getting the TD went BUT the lower Florida State's chances of responding went too. This edition assumes Auburn goes for it on the 4th down on what would be a single make-or-break play for championship.

What if KC Had Just Kneeled Out the 2nd Half?

With 13:39 left in the 3rd quarter, KC led IND 38-10. Obviously, 3rd grade math tells us IND needed at least 4 touchdowns just to tie the game at this point. A little more arithmetic might illustrate just how badly KC bungled this game.

There were a total of 1,519 seconds left in the game. KC can burn 40 seconds between plays and 6 seconds during a typical play just by calling a super safe run (that stays in bounds) or even a kneel. Even if KC doesn't try to convert a single first down, they can burn 144 seconds on a series. However, IND can use its 3 timeouts to make one series only take 24 seconds off the game clock.

Because IND is due to receive the kickoff at 13:39 in the 3rd, KC was guaranteed to have at least three possessions--one between each theoretical IND TD. That means that just by kneeling, KC can burn a total of 456 seconds (7:36) off the game clock, which leaves a total of 1,207 seconds of game time (20:07) and no timeouts for IND to score 4 touchdowns.

Chargers Courageous Call & Playoff-Clinching Drive

Despite the controversy surrounding an illegal defense on the Chiefs' missed field goal at the end of regulation, the San Diego Chargers defied odds and clinched a postseason berth on Sunday. In overtime, Philip Rivers orchestrated a 17-play, nine-and-a-half minute field goal drive to start the extra quarter that ultimately sealed their win. The length of the drive, in this case, is just as important as the outcome as San Diego could advance with either a win or a tie.

Using our Markov model, let's take a look at the drive. Keep in mind, the model is best used for a standard drive when time and score differential would not greatly affect decision-making or play-calling. Since this was the opening drive of overtime, those standards will predominantly hold true, although not perfectly given the leverage of the situation.

PIT Should Not Have Scored the TD

With the score tied and 1:51 to play, PIT had a 1st and 10 on the GB 17 yard line. In many circumstances a team can run down the clock and kick a short FG to win the game. PIT was near the 'Field Goal Choke Hold' zone, when it's better for the offense not to score a TD and better for the defense to allow a TD. But fortunately for GB, they had all 3 of their timeouts, and could be assured of getting the ball back with 1:27 to play if they made a stop and forced the FG. So with 3 timeouts remaining, the numbers say it never makes sense for a defense to intentionally allow the TD.

But GB jumped offside on the FG attempt, and gave PIT a 1st down and goal from the 5 with 1:35 to play. Now GB had only 1 timeout left, and it would have certainly made sense for PIT to refrain from scoring the TD, burn time off the clock, and kick an easy FG for the win.

The chart below illustrates when a defense would prefer to allow a TD. The black diamond represents the state of the game at the 1st and goal mark assuming PIT does not score a TD. The black line shows the win probability of the defense if they allow the TD.

Did Shanahan Make the Right Decision to Go for Two?

Following a TD to pull within an extra point of tying the game with just seconds left to play, WAS head coach Mike Shanahan elected to attempt a two-point conversion to win rather than enter overtime. It’s not fashionable to defend Shanahan around DC these days, but I think this was the right decision for a couple reasons.

Philbin Shows Up Phil

In a huge division game with significant playoff implications, the Dolphins trailed the Patriots 20-17 with a little over three minutes remaining and three timeouts in their back pocket. Facing a 4th-and-5 from their own 45-yard line, Jim Nantz asked Phil Simms whether he would punt in that situation, to which Simms replied "Absolutely, punt it!" The logic here is that with three timeouts remaining, the Dolphins could stop the Patriots and force a three-and-out, getting the ball back with another chance to drive down and tie or win the game.

Let's look at the baseline win probability numbers:

Payton Was Right to Decline

At least according to Expected Points, he was.

Here's the situation: At the beginning of the 3rd quarter against CAR, NO had a 1st and 10 at their own 16-yard line. They threw for a 7-yard gain, setting up a 2nd and 3 from their 23. But CAR was flagged for defensive holding, which would have given NO 5 yards and an automatic first down at their 21. NO head coach Sean Payton declined the penalty to the bafflement of many including the tv announcers.

The game did not hinge on this decision by any stretch. But it's worth taking a look at. The EP model is probably the right tool for the job in this case because it gives a much finer level of precision to down/distance/yd-line situations than the WP model or other approaches.

Using the hand-dandy WP calculator tool (which as a bonus is also an EP calculator), here are the relevant numbers:

Seahawks Stumble, Should Have Allowed TD

In one of the most anticipated games of the week, the San Francisco 49ers took over down 17-16 to the Seattle Seahawks with 6:20 remaining. After a huge Frank Gore 51-yard run, the Niners lined up for a 1st-and-Goal from the 7-yard line with 2:39 remaining. Seattle had no timeouts remaining. Should the Seahawks have tried to intentionally allow the Niners to score a touchdown? Let's look at Brian's graph for this situation in his intentional TD study:

Saban's Hyperbola: Analyzing Alabama's Long FG Attempt

Way late to the party here, but let's do this because it's so interesting. As every football fan on the planet knows, Alabama attempted a 57-yd FG with 1 sec to play in regulation against Auburn. The kick fell short and was returned for a stunning game-winning TD. The consensus analysis seems to be that the FG attempt wasn't necessarily a bad decision, but the big mistake was that Alabama was not prepared with appropriate personnel to cover a potential return.

Let's look at the FG decision more closely. I won't use the WP model, but instead apply some math and logic. There were three options for Alabama:

1. Kneel
2. Hail Mary
3. Attempt the FG

Let's make some assumptions. First, OT is a 50/50 proposition. Alabama was favored in this game, but Auburn was playing strong. Plus, OT is a bit of a dice roll to begin with. Second, Hail Marys (Maries, Mary's?) from that range are probably no more successful in college than they are in the pros, which is around 5%. Lastly, for the sake of the argument, let's say there is zero chance of a defensive TD return on the Hail Mary.

We don't really know the probability of a successful FG attempt or the probability of a successful return or block & return from a range like that, especially in college ball from a kicker without many attempts. But let's set that aside for a moment.

Was Belichick Right to Take the Wind in OT?

I was surprised when Bill Belichick chose to take the second possession (and risk no possessions) in OT against Peyton Manning and a team that had scored 31 points in four quarters. Although the new OT format mitigates the advantage of the team with first possession, it's still there to the tune of about 56% to 44%.

The advantage of wind must have felt fairly strong to Belichick. His team captains thought he was crazy. At the time, it was impossible to tell from the comfort of my sofa how bad the wind was, but I was curious if we could see the effect statistically.

McCarthy Makes New OT Mistake

Kicking a field goal on the two is like kissing your sister.

I hate ties. Both as a fan and as a stat cruncher. But I hate them a lot more as a stat cruncher.
— Brian Burke (@Adv_NFL_Stats) November 24, 2013

I could not have said it better myself. Nothing is more annoying when trying quantify a season and run simulations than coding for a tie. The Packers and Vikings tied 26-26 on Sunday after both teams kicked field goals in overtime. On the opening drive, Matt Flynn led the Packers to the Vikings 2-yard line before Mike McCarthy decided to kick a field goal, sending the game into the "chance to match down three" format. In the new overtime format, was this the optimal decision?

Did Marvin Lewis Make the Right Call on 4th Down in OT?

After a freak Hail Mary TD to tie the game, Cincinnati won the coin flip to start OT and drove to the BAL 33. Facing a 4th and 2, Lewis decided to for the conversion rather than attempt a 51-yard FG or punt.

Under the old sudden death OT rules, every coach would have undoubtedly attempted the FG in that situation. But under the new OT format, things have changed. Because the opponent gets an opportunity to match a first-possession FG, or to trump it with a TD, long FG attempts are not the percentage play.

If you make it, you've given your opponent all four downs to cruise down the field to respond. Plus, there is no urgency like in other four-down desperation drives because the clock is not a factor. And if you miss the FG, you've given the ball to your opponent in decent field position while triggering sudden death rules. Now, an opponent FG would end the game.

Trestman's 4th and Inches Call

I received a few requests to analyze Marc Trestman's decision to go for it on 4th and inches from his own 32, up by 4 with 7:50 to play in the 4th quarter. So here goes:

Punting would hand GB the ball at or around their own 20 yard line, worth 0.71 WP for CHI.

A successful conversion means a 1st and 10 at CHI's own 33, which would give CHI a Win Probability (WP) of 0.79. And a failed conversion attempt gives GB the ball at the CHI 32, worth 0.51 WP for CHI. [That's a relatively high-leverage situation--a potential swing of 0.28 WP.]

The break-even conversion probability (x) required to make it worthwhile to go for the conversion can be found by setting the value of the punt equal to the total value of the conversion attempt:

Slate: Punt From the Opponent's 26?

I make my first appearance this season at Slate to propose that DAL may have been better off punting from the DET 26 than attempting a 44-yd FG. If DAL could have pinned DET at or inside their own 10, the numbers suggest punting might have been preferable to going for it or trying the FG. It may have even been preferable to making the FG. I also discuss DAL's holding penalty that was the start of the critical path toward an improbable comeback.

...But there’s an extra wrinkle. Strangely, Dallas would have preferred to keep Detroit within 3 points rather than extend its lead to 6. When desperate teams like the Lions with no timeouts remaining get into the outer rim of field goal range, they send in the field goal unit for a long-range attempt. This is an irrational decision, one I discovered the very first time I began looking at win probability numbers. Rather than try to win the game, teams in this situation settle for a tie—or rather, an attempted tie. Even if the field goal attempt is good, it only buys a 50–50 shot at the win in overtime...

Packers' Perfect Third Quarter

After a grown-man run from Adrian Peterson to end the first half, the Green Bay Packers opened up the second half up only a touchdown to the dismal Vikings, 24-17. Aaron Rodgers led the Pack on a 16-play, 80-yard touchdown drive that lasted over eight minutes. During the march, Green Bay converted on three third downs and a fourth down. Let's look at the progression of the drive using our Markov model:

Was BAL's Onside Kick Attempt Smart?

Trailing with 13 minutes left in the 4th quarter vs. PIT, BAL kicked a FG to make the score 13-9. BAL then attempted a surprise onside kick attempt, which was unsuccessful. What do the numbers say?

Surprise onside kick attempts are generally worth the gamble. Based on the Expected Points model, a normal kickoff is typically worth -0.42 EP, as the opponent has a 1st down at about their own 22. A recovered onside kick (1st down at one's own 45) is worth 1.77 EP. A failed onside kick (opponent's 1st down a one's own 45) is worth -2.36 EP. The break-even recovery probability would therefore be:

Jets 'Push' Their Luck

After forcing overtime, the Jets stopped the Patriots on their first drive, reverting to the old OT format - sudden death. Geno Smith and the Jets moved downfield before being stopped for a 4th-and-7 from the New England 38. Rex Ryan had three viable options here, keeping in mind that the next score wins: Kick a low-probability (40% league-wide) 55-yard field goal, attempt to convert a low-probability (42% league-wide) 4th-and-7, or punt the ball deep and risk Tom Brady leading a game-winning drive.

The Jets elected to attempt the field goal. Nick Folk missed wide left, but in a crazy turn of events, New England was penalized 15-yards for an unsportsmanlike conduct "pushing" penalty. Before we get to the penalty, let's talk about the decision. While I almost always advocate going for it in no-man's land, in this situation, I was leaning toward the punt.

For this analysis, I used a combination of my Markov probabilities as well as Brian's overtime win probabilities.

Dolphins Should Have Intentionally Allowed the TD

Leading by a point, the MIA defense was on its heals as BUF drove into FG range with less than 3 minutes to play. With all 3 timeouts, MIA must have been feeling pretty good about its chances of getting the ball back with some time to retake the lead.

BUF ran the ball, forcing MIA to use its timeouts. With one timeout remaining and 2:37 to play, BUF faced a 3rd and 4 at the MIA 28. BUF ran the ball for a 10-yard gain, earning a fresh set of downs and forcing MIA to use its last timeout.

At this point, MIA would have preferred to allow BUF to score a TD. The odds favored scoring a TD of their own in response. Accordingly, BUF should have preferred to take a knee rather than score then.

Here is the chart, built using this research. The red dot is where MIA found itself. The dotted black line is the Win Probability curve for allowing the TD, and for reference, the teal line is the 20 yd line.

Analyzing the Patriots' 4th Down Calls

Kurt Bullard is a freshman at Harvard and a first year member of the Harvard Sports Analysis Collective. He intends to major in either Economics or Statistics. Go 'Cuse.

This Sunday, the Patriots found themselves down six points to the Saints with only 1:13 left in the game. In that span, Tom Brady was able to lead a drive down the field, connecting on a 17-yard pass to Kenbrell Thompkins in the left corner of the end zone to complete one of Boston’s two major come-from-behind wins of the day.

Hidden by the final drive were two controversial fourth down calls in the fourth quarter that happened at key moments in the game. The first came with 8:34 remaining in the game when the Patriots held a 20-17 lead over the Saints. Faced with a 4th-and-goal at the five yard line, New England opted to kick a field goal rather than try to score a touchdown to go up two scores.

The win probability and expected point value each suggest different optimal decisions. The expected point formula suggests that the Patriots ought to have gone for it, while the win probability calculator says otherwise, albeit by a slight margin.

Bengals Punt Away Regulation

Heading into the fourth quarter, the Cincinnati Bengals led the Buffalo Bills handily, 24-10. With just over 10 minutes left, the Bills drove down the field and found themselves in a 4th-and-8 from the 22-yard line. With that much time left on the clock, it is safe to assume that most NFL coaches would kick the field goal. In fact, with backup Thad Lewis under center, the probability of coaches kicking the field goal would increase. The Bills made a gutsy call and went for it, hitting Scott Chandler for a 22-yard strike and scoring. The numbers say it’s a toss up:

Bengals' Big Time Drive

A week after losing to the Cleveland Browns, the Cincinnati Bengals were tasked with taking down the undefeated New England Patriots in Week 5. The game was not enjoyable to watch and had a baseball score of 6-3 heading into the fourth quarter. Needing to separate themselves from the Patriots, the Bengals put together a 15-play, 93-yard, nearly eight minute drive, ultimately scoring a touchdown to go up two scores, 13-3.

Using our Markov model, we can look at the evolution of the drive:

When Should DAL Have Allowed DEN to Score a TD?

Once Demaryius Thomas crossed the line to gain at the DAL 14 yard line with 1:50 to play, the DAL defense should have intentionally allowed the TD. With 2 timeouts and 1:40+ to play, they would have had a better chance of winning than allowing DEN to choke the life out of them and kick an easy FG for the win.

That's based on this analysis of when teams should intentionally allow a TD when tied.

Aften Thomas gained a 1st down, DAL correctly called a timeout setting up a 1st down and 10 at the DAL 11 with 1:49 to play. This chart illustrates DAL's chances of winning based on field position and time remaining with 1 timeout left.

Denver over Dallas: Instant Analysis

Sunday's game was a hard fought contest of wills, but in the end Denver pulled out the much-needed victory over Dallas. Question marks remain for both teams, however. Denver was finally able to find their identity and stick with what works. Although it's not time to panic yet in Dallas, they're going to need to find a way to get it done before it's too late.

Peyton Manning silenced his critics with a solid, although imperfect, outing. "For me it's not about the individual records," he said after the game. "It's all about the team. I'm just glad we were able to go out there and get the win today," he added. Peyton Manning needs to be careful not to look past next week's opponent. Their upcoming match-up will be nationally televised in prime time, a classic trap game.

Tony Romo looked confident at times, but eventually let the game slip out of his grasp. He said after the game, "We didn't bring our A-game today. We didn't step up when we needed to and make the plays we needed to make." Tony Romo showed a lot of courage in the face of adversity on Sunday. Now he needs to put it all together for Dallas to get back among the top teams in the league.

Dallas just can't continue to make the kinds of mistakes they made at this level of the sport. Dallas fans had higher hopes for the defense this season, but if they keep playing at this level significant changes will be needed in the offseason. The offense's performance was also less than inspiring, stalling numerous times in opponent territory.

Dallas lost that game more than Denver won it. Everyone knows football is a game of emotion, and clearly Denver showed they had the passion to come out on top. You could feel the electricity in the stadium from kickoff to the final whistle. It was a game for the ages.

Don't like this column? Generate your own with the ANS Sports Column Generator!

What Are You Doing, Chip?

Cards on the table, I'm a huge Eagles fan. As an NFL stats nerd, I could not have been more excited for Chip Kelly to make the transition to the big leagues. While I did not expect him to immediately institute his Oregon trademarks, I did expect to see him going for it more often on fourth down, especially in situations where the numbers called for it -- and generally, making decisions to maximize the Eagles win probability.

It's four weeks into the season, and too many times I've asked my TV, "What are you doing, Chip?" Today against the Broncos, there were a couple of questionable decisions. Down 14-3, the Eagles were moving the ball very well to start the game. Vick and company strung together a 15-play drive that ended up with a 4th-and-4 from the Broncos 7-yard line. Using our Markov model, we can look at the progression of the drive:

Rex Ryan Runs Out of Challenges

Early in the 4th quarter of Sunday's BUF-NYJ game, the Jets led by 8. BUF QB EJ Manuel apparently lost a fumble, but the officials ruled him down. Had NYJ recovered, they would have had the lead and a 1st and 10 in BUF territory, only about 5 yards from realistic FG (attempt) range, with about 13 minutes left to play. The only problem was that NYJ head coach Rex Ryan had no more challenges, having burned them both just minutes prior to the fumble that wasn't.

Let's examine the leverage of each challenge using the Win Probability (WP) model. 

Jason Garrett Should Have Punted

by Matt Meiselman

Matt has been helped me crunch some numbers this off-season. He is graduate of the University of Maryland with a degree in broadcast journalism. He's originally from New Jersey, but loves New York sports. Matt aspires to work in sports media and has a passion for sports statistics. 

We're trying hard not to be broken records on 4th down decisions, but this one is special. -BB

Jason Garrett is not typically regarded as a savvy 4th down decision maker, and his clock management skills aren’t the greatest either. Garrett consistently makes blunders in way too many of his strategic decisions. But even after all the mistakes he’s made as the Cowboys head coach, he somehow found a way to top every one of them with a single mistake in Week 2 against the Chiefs.

With 3:50 to play in the 4th quarter, Dallas trailed Kansas City 17-13. It was the Cowboys’ ball, but they faced a 4th and 10 from the Chiefs 35. They held all three of their timeouts and undoubtedly still had a decent opportunity to come away with a win. Garrett had a decision to make: should he go for it? Punt? Or a kick a field goal?

Based on the 4th down calculator, the Cowboys chose the worst of the three options, and it wasn’t even close.

Bucs Botch Fourth Down Call

With 1:10 remaining in the fourth quarter, Tampa Bay led 14-13 on the Saints' 29-yard line. Facing a 4th-and-3, with an 82% chance to win the game, Greg Schiano elected to attempt the 46-yard field goal - about a 64% proposition. If the Bucs hit the field goal, they go up four points, increasing their win probability to 86%. With a miss, the win probability drops to 74% and Drew Brees gets the ball on the 36, needing only a field goal to win. 

Schiano also had two other options: punt the ball and pin the Saints deep or go for it. Below are the numbers:

Saints Slip Up, Still Win

Sean Payton is back in the decision-maker chair for the Saints after his one-year sabbatical, and he is typically known as one of the more analytical coaches. The Saints pass more often than other teams, they go for it on fourth down more often, and even try the occasional surprise onside kick.

On Sunday, Payton faced a couple of fourth down decisions, testing his analytical mind. On their second drive, the Saints ran the ball on 4th-and-1 from their own 47-yard line -- a sizable gamble to the traditional football mind. Mark Ingram was stuffed and the Falcons would turn good starting field position into a field goal. This was definitely the right decision, despite the outcome (+0.84 EP going for it vs -0.30 EP with a punt).

The much bigger decision, though, came with 3:30 left in the game up three. The Saints faced a 4th-and-2 from the 4-yard line after a 5-yard completion to Lance Moore. Drew Brees lined up to attempt a 4th-down conversion but Payton ultimately decided to take a timeout to think things over. After the timeout, the Saints came out in field goal formation, made the chip shot and went up 23-17.

So, what do the numbers say?

Slate/Deadspin: The Little Things that Cost the 49ers

My analysis for SB47 is up over at Deadspin and Slate. I took a look at some of the more interesting decisions from the game--the fake FG, the intentional safety, and a couple 4th downs. But the biggest decisions in the game turned out to be the littlest ones. If there were ever a case study in the importance of saving your 2nd half timeouts, this game should be it.

Super Bowl Statistical Matchups

Here is your one-stop shop for all things advanced stats for SB 47. Team efficiency stats. Advanced team stats. Top individual performers from each team.

Don't Overlook the Effect of Penalties

One of the more overlooked aspects of team performance is the tendency for being penalized. Penalty rate, defined as penalty yards per snap is one of the more reliably stable team stats. Compared to things like running or passing efficiency, the absolute size of penalty rate's effect is small, but because it tends to be consistent, it can be fairly predictive.

This Sunday's conference championship games feature teams on opposite sides of the penalty spectrum. ATL has by far the league's lowest team penalty rate at 0.21 penalty yards per snap. For context, the league average is 0.41, and the next best team is NYG at 0.29. ATL is 3.9 standard deviations better than the 2012 mean! SF is third worst in the league 0.46 penalty yds per snap.

On the other side of things is BAL. They're averaging 0.53 penalty yards per snap, the league's worst rate. That's 1.9 standard deviations worse than the mean. NE is tied for 6th best, at 0.39 penalty yds per snap.

The Seahawks' 4th and 1

Pete Carroll is getting a lot of grief today about going for it on 4th and 1 from the ATL 11 in the 2nd qtr. It was the slam-dunk right call, which I'll explain below. But worse, much of the criticism is completely misguided. I would, however, knock SEA for the play call.

I keep reading and hearing about how the failed 4th down attempt 'came back to bite Seattle' or that the decision cost 3 points in a game decided by 2. First of all, that's complete outcome bias. Expunge that kind of thinking from your head. Had SEA converted, the same people who are now critical would be praising Carroll for his courage. Second, you can't simply append 3 extra points onto the final score and say that would have changed the outcome. You never know how the game may have unfolded had Carroll decided on the FG attempt. Both teams would have played at different risk levels and at different paces had the score been different, particularly in the endgame.

I've refrained from the cookie-cutter 'should have gone for it, coach' posts this season for fear of becoming a one-trick pony. (Especially now that everyone can do the same thing himself with the 4th Down Calculator.) But this was a critical call in an important game, so I'll indulge. Here are the numbers on the 4th down itself. Remember, results are based on league average baseline numbers. It was 4th and 1 on the ATL 11 with 5:38 in the 2nd qtr. ATL was up 13-0.

Coaches Bring Passiveness To Wild Card Weekend

I'm never quite sure what decision-making trends to expect out of the NFL playoffs. It seems any decision can be justified by the playoffs. "It's the playoffs," one of the exalted keepers of the true knowledge can say. "You have to leave it all on the line," he says as the coach keeps the offense on the field for a fourth and goal.

But, he could just as easily say, "You don't have a choice here. You have to live to fight another day." The field goal team trots out for a 20-yard chip shot instead.

The field goal teams were out in force for Wild Card Weekend. Presented with 27 fourth downs inside the opponent's 40 yard line, teams kicked 17 field goals, punted twice, and went for it just eight times. Of the 17 field goal attempts, only eight were the optimal win expectancy choice according to the 4th down calculator. All told, coaches left 0.24 of win expectancy and 6.3 expected points on the table with these decisions.

Overall, teams saw 66 fourth down plays and made the optimal decision 49 times. Only one of the 11 decisions to go for it was suboptimal (by Washington when they were already down by 10 late in the game) and seven of the 33 punts were as well. The biggest whiffs were typically in field goal situations, but to the coaches' credit, the kickers were sharp: they combined to convert 16 of the 17 field goals on the week.

Still, there were a few calls worth questioning even given the true kicks. After making a borderline call to go for it on fourth-and-5 from the 34 -- probably the right call given the unreliability of Mason Crosby this season -- and succeeding, the Packers kicked on a fourth-and-goal from the one yard line with 3:25 to go in the 2nd quarter. The Seahawks, up by seven against the Redskins, chose to kick on fourth-and-goal from the 4 with 5:32 to go in the game.

Cowboys Kick (Twice), But Still Win

In one of the more intriguing match-ups of the weekend, Tony Romo and the Cowboys received the ball in a tie game just over midfield with 0:47 seconds remaining. After two incompletions and one pass to Jason Witten, the Cowboys faced a 4th-and-4 at the Steelers 43-yard line. With 0:32 seconds to go and one timeout left, the Cowboys trot out their field goal team for a miraculous 60-yard game-winning kick. Both teams would call timeout and Jason Garrett would ultimately decide to punt instead, essentially assuring overtime. Let's think about this decision generally before looking at the frequency specifics.

If you punt, you are conceding to play overtime, giving your team a 50/50 shot to win the game. If you kick a field goal from that range, the probability that it goes in is at best the same as the probability that you miss and your opponent can score before the end of regulation, giving you a 50/50 chance to win optimistically. If you go for it and convert, you will have a much higher probability of making the field goal even with time constraints -- if you fail, you are pretty much in the same situation as a missed field goal.

Niners Nine-Minute Drive

Up 28-21 with the ball nearing the end of the third quarter, the Niners had a 78% of winning the game over the streaking Saints. That 78% does not account for the prolific Saints offense, though. What followed was a nine-and-a-half minute drive, lasting deep into the fourth quarter, draining precious time that Drew Brees would need in order to make a comeback. Any Saints fan -- or someone whose fantasy team depends on the Saints offense like myself -- could not have been more frustrated watching the Niners rumble down the field over the course of 17 plays and 85 yards.

Let's take a look at the evolution of the drive using our Markov model:

One Play Remaining before the Half on the Goal Line

IND QB Andrew Luck spiked the ball with 1 sec remaining in the 2nd quarter, bringing up a 3rd and goal from the 1-yd line. Without a moment of hesitation, acting head coach Bruce Arians ran in the FG unit for the chip-shot. The FG was good and IND took at 13-6 lead over BUF into the locker room. Was this the smart call?

Let's set aside the score and look at the general case. It's a special situation because there is no subsequent kickoff. Instead of being worth 2.7 Expected Points (EP), a FG is worth a full 3 EP. And a TD would be worth a full 7 EP instead of 6.7. The offense would take the full value of the score.

The expected value of each choice is straightforward. It's just the probability of success * the value of the score. In the case of the FG it would be:

How Much Did Jim Schwartz's Attempted Challenge Cost the Lions?

Had Schwartz not thrown the challenge flag on Forsett's run, the play would have been reviewed and certainly overturned. That would mean a 3rd and 2 for HOU on their own 27 with 6:40 or so in the 3rd quarter. Down by 10 points, that means at 0.18 Win Probability (WP) for HOU.

But because Schwartz threw the challenge flag on a play that would have been otherwise reviewed automatically, he received an unsportsmanlike penalty. The result was that the play was not reviewed by rule, and Forsett's TD stood. That gave DET a touchback up by 3, giving HOU a 0.35 WP.

That's a cost of 0.17 WP. It essentially doubled HOU's chances of winning at that point.

Panthers Punt Away The Win

On Saturday, I watched as my Northwestern Wildcats punted on 4th-and-1 from midfield, up three with 1:30 remaining. A conversion would have guaranteed a win, allowing the Cats to kneel down. By punting to Michigan State, I was certain NU would lose in dramatic fashion once again. Luckily, the Cats defense came up big, making what seemed like their first fourth-quarter stand of the year. Ron Rivera had an eerily similar decision against the Bucs this week.

With just over a minute left, the Panthers had the ball at the Tampa Bay 49, up 21-13. The Bucs had used all three of their timeouts, so a conversion wins the game. Not only do the Panthers have one of the best goal line backs in the league in Cam Newton, but they also have the human bowling ball, Mike Tolbert, who was born and bred for those situations.

First Falcons Drive & Botched Bears Field Goal

Andy Reid had never lost coming off a bye week in his head-coaching career. Matt Ryan did not care. Leading the Falcons on a 8 minute 44 second opening drive, Ryan and company marched 80 yards down the field over 18 plays before hitting Drew Davis on a 15-yard touchdown. Up 7-0, the Falcons were 72% favorites to win the game and that probability would never dip below that the rest of the day.

Here is a quick look at Matty Ice's first drive -- on which he went 6 for 7 for 62 yards -- using our Markov model: